{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第4阶段_第4讲_探索性数据分析(EDA)\n",
    "\n",
    "## 学习目标\n",
    "1. 理解EDA的核心思想和完整流程\n",
    "2. 掌握单变量分析的方法(数值型和分类型)\n",
    "3. 掌握双变量分析的方法(数值-数值、数值-分类、分类-分类)\n",
    "4. 学会使用多种可视化手段进行数据探索\n",
    "5. 能够发现数据中的模式、趋势、异常和关系\n",
    "6. 完成一个完整的EDA项目实战\n",
    "\n",
    "## 什么是探索性数据分析(EDA)?\n",
    "\n",
    "**探索性数据分析(Exploratory Data Analysis, EDA)**是数据分析的重要环节,目标是:\n",
    "\n",
    "### EDA的核心目的\n",
    "- 📊 **了解数据结构**: 数据量、字段类型、缺失情况\n",
    "- 🔍 **发现数据特征**: 分布、集中趋势、离散程度\n",
    "- 🎯 **识别异常值**: 极端值、错误数据\n",
    "- 🔗 **探索变量关系**: 相关性、因果关系\n",
    "- 💡 **生成业务洞察**: 发现有价值的业务模式\n",
    "- 🛠️ **为建模做准备**: 特征选择、数据清洗方向\n",
    "\n",
    "### EDA的六步流程\n",
    "\n",
    "```\n",
    "1. 数据概览 → 2. 单变量分析 → 3. 双变量分析 → 4. 多变量分析 → 5. 异常检测 → 6. 总结洞察\n",
    "```\n",
    "\n",
    "## Excel vs Pandas EDA对比\n",
    "\n",
    "| 功能 | Excel | Pandas + Matplotlib |\n",
    "|------|-------|---------------------|\n",
    "| 数据概览 | 手动查看 | df.info(), df.head() |\n",
    "| 统计摘要 | 数据透视表 | df.describe() |\n",
    "| 分布图 | 插入图表-直方图 | plt.hist(), sns.histplot() |\n",
    "| 箱线图 | 插入图表-箱线图 | plt.boxplot(), sns.boxplot() |\n",
    "| 散点图 | 插入图表-散点图 | plt.scatter(), sns.scatterplot() |\n",
    "| 相关分析 | CORREL函数 | df.corr(), sns.heatmap() |\n",
    "| 分组分析 | 数据透视表 | df.groupby() |\n",
    "| 自动化 | 手动重复 | 脚本一键生成 |\n",
    "| 数据量 | <10万行 | 百万级+ |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入必要的库\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from scipy import stats\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "# 设置中文显示\n",
    "plt.rcParams['font.sans-serif'] = ['Arial Unicode MS', 'SimHei']\n",
    "plt.rcParams['axes.unicode_minus'] = False\n",
    "\n",
    "# 设置seaborn样式\n",
    "sns.set_style('whitegrid')\n",
    "sns.set_palette('husl')\n",
    "\n",
    "# 设置显示选项\n",
    "pd.set_option('display.max_columns', None)\n",
    "pd.set_option('display.width', 1000)\n",
    "pd.set_option('display.float_format', '{:.2f}'.format)\n",
    "\n",
    "print(\"✅ 环境配置完成!\")\n",
    "print(f\"Pandas版本: {pd.__version__}\")\n",
    "print(f\"NumPy版本: {np.__version__}\")\n",
    "print(f\"Seaborn版本: {sns.__version__}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一、数据概览(Data Overview)\n",
    "\n",
    "### 案例背景:某电商平台用户行为数据\n",
    "\n",
    "我们将分析一个电商平台的用户购买行为数据,包含:\n",
    "- 用户基本信息(年龄、性别、会员等级)\n",
    "- 购买行为(购买金额、购买频次、客单价)\n",
    "- 用户活跃度(登录次数、浏览时长)\n",
    "- 满意度指标(评分、复购率)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建模拟的电商用户数据\n",
    "np.random.seed(42)\n",
    "\n",
    "n_users = 1000\n",
    "\n",
    "# 生成用户基本信息\n",
    "user_data = pd.DataFrame({\n",
    "    '用户ID': [f'U{str(i).zfill(5)}' for i in range(1, n_users+1)],\n",
    "    '年龄': np.random.normal(32, 10, n_users).clip(18, 65).astype(int),\n",
    "    '性别': np.random.choice(['男', '女'], n_users, p=[0.48, 0.52]),\n",
    "    '城市等级': np.random.choice(['一线', '二线', '三线', '四线'], n_users, p=[0.25, 0.35, 0.25, 0.15]),\n",
    "    '会员等级': np.random.choice(['普通', '白银', '黄金', '铂金', '钻石'], n_users, p=[0.4, 0.3, 0.15, 0.1, 0.05]),\n",
    "    '注册天数': np.random.poisson(lam=300, size=n_users).clip(30, 1000),\n",
    "    '月购买次数': np.random.poisson(lam=5, size=n_users).clip(0, 50),\n",
    "    '月消费金额': np.random.gamma(shape=2, scale=500, size=n_users).clip(0, 10000),\n",
    "    '月登录次数': np.random.poisson(lam=15, size=n_users).clip(1, 100),\n",
    "    '平均浏览时长': np.random.exponential(scale=20, size=n_users).clip(1, 120),  # 分钟\n",
    "    '客户评分': np.random.choice([1, 2, 3, 4, 5], n_users, p=[0.02, 0.05, 0.15, 0.45, 0.33]),\n",
    "    '是否复购': np.random.choice(['是', '否'], n_users, p=[0.65, 0.35])\n",
    "})\n",
    "\n",
    "# 根据会员等级调整消费金额(更真实)\n",
    "level_multiplier = {'普通': 0.7, '白银': 1.0, '黄金': 1.5, '铂金': 2.0, '钻石': 3.0}\n",
    "user_data['月消费金额'] = user_data.apply(\n",
    "    lambda row: row['月消费金额'] * level_multiplier[row['会员等级']], axis=1\n",
    ")\n",
    "\n",
    "# 计算客单价\n",
    "user_data['客单价'] = user_data['月消费金额'] / (user_data['月购买次数'] + 1)  # +1避免除零\n",
    "\n",
    "# 根据年龄段调整行为(年轻人活跃度更高)\n",
    "user_data.loc[user_data['年龄'] < 30, '月登录次数'] = (user_data.loc[user_data['年龄'] < 30, '月登录次数'] * 1.3).astype(int)\n",
    "\n",
    "# 添加一些缺失值(模拟真实场景)\n",
    "user_data.loc[np.random.choice(user_data.index, 30, replace=False), '平均浏览时长'] = np.nan\n",
    "user_data.loc[np.random.choice(user_data.index, 20, replace=False), '客户评分'] = np.nan\n",
    "\n",
    "# 添加一些异常值\n",
    "outlier_indices = np.random.choice(user_data.index, 5, replace=False)\n",
    "user_data.loc[outlier_indices, '月消费金额'] = np.random.uniform(15000, 30000, 5)\n",
    "\n",
    "# 保留两位小数\n",
    "user_data['月消费金额'] = user_data['月消费金额'].round(2)\n",
    "user_data['客单价'] = user_data['客单价'].round(2)\n",
    "user_data['平均浏览时长'] = user_data['平均浏览时长'].round(2)\n",
    "\n",
    "print(\"✅ 数据集创建完成!\")\n",
    "print(f\"数据集规模: {user_data.shape[0]}行 × {user_data.shape[1]}列\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 第一步:快速查看数据\n",
    "print(\"=\"*100)\n",
    "print(\"1️⃣ 数据前5行预览\")\n",
    "print(\"=\"*100)\n",
    "print(user_data.head())\n",
    "\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"2️⃣ 数据后5行预览\")\n",
    "print(\"=\"*100)\n",
    "print(user_data.tail())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 第二步:数据基本信息\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"3️⃣ 数据集基本信息\")\n",
    "print(\"=\"*100)\n",
    "print(user_data.info())\n",
    "\n",
    "print(\"\\n💡 关键发现:\")\n",
    "print(f\"  - 总记录数: {len(user_data)} 条\")\n",
    "print(f\"  - 总字段数: {len(user_data.columns)} 个\")\n",
    "print(f\"  - 数值型字段: {len(user_data.select_dtypes(include=[np.number]).columns)} 个\")\n",
    "print(f\"  - 对象型字段: {len(user_data.select_dtypes(include=['object']).columns)} 个\")\n",
    "print(f\"  - 缺失值字段: {user_data.isnull().any().sum()} 个\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 第三步:缺失值分析\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"4️⃣ 缺失值统计\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "missing_stats = pd.DataFrame({\n",
    "    '缺失数量': user_data.isnull().sum(),\n",
    "    '缺失比例(%)': (user_data.isnull().sum() / len(user_data) * 100).round(2)\n",
    "})\n",
    "missing_stats = missing_stats[missing_stats['缺失数量'] > 0].sort_values('缺失数量', ascending=False)\n",
    "\n",
    "if len(missing_stats) > 0:\n",
    "    print(missing_stats)\n",
    "    print(\"\\n⚠️ 需要处理的缺失值:\")\n",
    "    for col, row in missing_stats.iterrows():\n",
    "        print(f\"  - {col}: {int(row['缺失数量'])}个缺失 ({row['缺失比例(%)']}%)\")\n",
    "else:\n",
    "    print(\"✅ 无缺失值!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 第四步:数据统计摘要\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"5️⃣ 数值型变量统计摘要\")\n",
    "print(\"=\"*100)\n",
    "print(user_data.describe().T)  # 转置显示更清晰\n",
    "\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"6️⃣ 分类型变量统计摘要\")\n",
    "print(\"=\"*100)\n",
    "print(user_data.describe(include='object').T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 第五步:数据分布快速可视化\n",
    "fig, axes = plt.subplots(2, 2, figsize=(15, 10))\n",
    "\n",
    "# 年龄分布\n",
    "axes[0, 0].hist(user_data['年龄'], bins=30, color='skyblue', edgecolor='black', alpha=0.7)\n",
    "axes[0, 0].axvline(user_data['年龄'].mean(), color='red', linestyle='--', linewidth=2, label=f'均值={user_data[\"年龄\"].mean():.1f}')\n",
    "axes[0, 0].set_title('用户年龄分布', fontsize=13, fontweight='bold')\n",
    "axes[0, 0].set_xlabel('年龄', fontsize=11)\n",
    "axes[0, 0].set_ylabel('用户数', fontsize=11)\n",
    "axes[0, 0].legend()\n",
    "axes[0, 0].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 月消费金额分布\n",
    "axes[0, 1].hist(user_data['月消费金额'], bins=40, color='coral', edgecolor='black', alpha=0.7)\n",
    "axes[0, 1].axvline(user_data['月消费金额'].median(), color='green', linestyle='--', linewidth=2, label=f'中位数={user_data[\"月消费金额\"].median():.0f}')\n",
    "axes[0, 1].set_title('月消费金额分布', fontsize=13, fontweight='bold')\n",
    "axes[0, 1].set_xlabel('月消费金额(元)', fontsize=11)\n",
    "axes[0, 1].set_ylabel('用户数', fontsize=11)\n",
    "axes[0, 1].legend()\n",
    "axes[0, 1].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 会员等级分布\n",
    "level_counts = user_data['会员等级'].value_counts().reindex(['普通', '白银', '黄金', '铂金', '钻石'])\n",
    "axes[1, 0].bar(level_counts.index, level_counts.values, color=['gray', 'silver', 'gold', 'skyblue', 'pink'], edgecolor='black')\n",
    "axes[1, 0].set_title('会员等级分布', fontsize=13, fontweight='bold')\n",
    "axes[1, 0].set_xlabel('会员等级', fontsize=11)\n",
    "axes[1, 0].set_ylabel('用户数', fontsize=11)\n",
    "axes[1, 0].grid(axis='y', alpha=0.3)\n",
    "for i, v in enumerate(level_counts.values):\n",
    "    axes[1, 0].text(i, v + 10, str(v), ha='center', fontsize=10)\n",
    "\n",
    "# 城市等级分布\n",
    "city_counts = user_data['城市等级'].value_counts().reindex(['一线', '二线', '三线', '四线'])\n",
    "colors_city = ['red', 'orange', 'yellow', 'green']\n",
    "axes[1, 1].pie(city_counts.values, labels=city_counts.index, autopct='%1.1f%%', colors=colors_city, startangle=90)\n",
    "axes[1, 1].set_title('城市等级分布', fontsize=13, fontweight='bold')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 数据概览总结:\")\n",
    "print(f\"  - 年龄集中在{user_data['年龄'].quantile(0.25):.0f}-{user_data['年龄'].quantile(0.75):.0f}岁\")\n",
    "print(f\"  - 月消费金额中位数{user_data['月消费金额'].median():.2f}元,存在高消费异常值\")\n",
    "print(f\"  - 普通会员占比最高({level_counts['普通']/len(user_data)*100:.1f}%),钻石会员最少({level_counts['钻石']/len(user_data)*100:.1f}%)\")\n",
    "print(f\"  - 用户主要来自一二线城市({(city_counts['一线']+city_counts['二线'])/len(user_data)*100:.1f}%)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二、单变量分析(Univariate Analysis)\n",
    "\n",
    "单变量分析关注**单个变量的分布特征**,分为:\n",
    "- **数值型变量**: 分布、集中趋势、离散程度、偏态\n",
    "- **分类型变量**: 频数、占比、众数\n",
    "\n",
    "### 2.1 数值型变量分析"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 选择一个关键数值变量进行深入分析:月消费金额\n",
    "target_var = '月消费金额'\n",
    "\n",
    "print(\"=\"*100)\n",
    "print(f\"📊 深入分析:{target_var}\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 基础统计\n",
    "print(\"\\n【基础统计指标】\")\n",
    "print(f\"  数量:     {user_data[target_var].count()}\")\n",
    "print(f\"  均值:     {user_data[target_var].mean():.2f} 元\")\n",
    "print(f\"  中位数:   {user_data[target_var].median():.2f} 元\")\n",
    "print(f\"  众数:     {user_data[target_var].mode()[0]:.2f} 元\")\n",
    "print(f\"  标准差:   {user_data[target_var].std():.2f} 元\")\n",
    "print(f\"  最小值:   {user_data[target_var].min():.2f} 元\")\n",
    "print(f\"  最大值:   {user_data[target_var].max():.2f} 元\")\n",
    "print(f\"  极差:     {user_data[target_var].max() - user_data[target_var].min():.2f} 元\")\n",
    "\n",
    "# 分位数\n",
    "print(\"\\n【分位数分析】\")\n",
    "for q in [0.1, 0.25, 0.5, 0.75, 0.9, 0.95, 0.99]:\n",
    "    print(f\"  {int(q*100)}%分位数: {user_data[target_var].quantile(q):.2f} 元\")\n",
    "\n",
    "# 分布形态\n",
    "skewness = user_data[target_var].skew()\n",
    "kurtosis = user_data[target_var].kurt()\n",
    "print(\"\\n【分布形态】\")\n",
    "print(f\"  偏度(Skewness):  {skewness:.3f} ({'右偏' if skewness > 0 else '左偏' if skewness < 0 else '对称'})\")\n",
    "print(f\"  峰度(Kurtosis):  {kurtosis:.3f} ({'尖峭' if kurtosis > 0 else '平坦' if kurtosis < 0 else '正态'})\")\n",
    "\n",
    "# 变异系数\n",
    "cv = user_data[target_var].std() / user_data[target_var].mean() * 100\n",
    "print(f\"  变异系数(CV):    {cv:.2f}% ({'高波动' if cv > 50 else '中波动' if cv > 25 else '低波动'})\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 月消费金额的多角度可视化\n",
    "fig, axes = plt.subplots(2, 3, figsize=(18, 10))\n",
    "\n",
    "# 1. 直方图\n",
    "axes[0, 0].hist(user_data[target_var], bins=50, color='steelblue', edgecolor='black', alpha=0.7)\n",
    "axes[0, 0].axvline(user_data[target_var].mean(), color='red', linestyle='--', linewidth=2, label='均值')\n",
    "axes[0, 0].axvline(user_data[target_var].median(), color='green', linestyle='--', linewidth=2, label='中位数')\n",
    "axes[0, 0].set_title('直方图(Histogram)', fontsize=12, fontweight='bold')\n",
    "axes[0, 0].set_xlabel('月消费金额(元)', fontsize=10)\n",
    "axes[0, 0].set_ylabel('频数', fontsize=10)\n",
    "axes[0, 0].legend(fontsize=9)\n",
    "axes[0, 0].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 2. 核密度图(KDE)\n",
    "user_data[target_var].plot(kind='kde', ax=axes[0, 1], color='purple', linewidth=2)\n",
    "axes[0, 1].axvline(user_data[target_var].mean(), color='red', linestyle='--', linewidth=2, label='均值')\n",
    "axes[0, 1].set_title('核密度图(KDE)', fontsize=12, fontweight='bold')\n",
    "axes[0, 1].set_xlabel('月消费金额(元)', fontsize=10)\n",
    "axes[0, 1].set_ylabel('密度', fontsize=10)\n",
    "axes[0, 1].legend(fontsize=9)\n",
    "axes[0, 1].grid(alpha=0.3)\n",
    "\n",
    "# 3. 箱线图\n",
    "bp = axes[0, 2].boxplot(user_data[target_var], vert=True, patch_artist=True,\n",
    "                         boxprops=dict(facecolor='lightgreen', alpha=0.7),\n",
    "                         medianprops=dict(color='red', linewidth=2),\n",
    "                         flierprops=dict(marker='o', markerfacecolor='red', markersize=5, alpha=0.5))\n",
    "axes[0, 2].set_title('箱线图(Boxplot)', fontsize=12, fontweight='bold')\n",
    "axes[0, 2].set_ylabel('月消费金额(元)', fontsize=10)\n",
    "axes[0, 2].set_xticklabels(['月消费金额'])\n",
    "axes[0, 2].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 4. 小提琴图(Violin Plot)\n",
    "parts = axes[1, 0].violinplot([user_data[target_var].dropna()], positions=[0], widths=0.7, showmeans=True, showmedians=True)\n",
    "axes[1, 0].set_title('小提琴图(Violin Plot)', fontsize=12, fontweight='bold')\n",
    "axes[1, 0].set_ylabel('月消费金额(元)', fontsize=10)\n",
    "axes[1, 0].set_xticks([0])\n",
    "axes[1, 0].set_xticklabels(['月消费金额'])\n",
    "axes[1, 0].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 5. Q-Q图(检验正态性)\n",
    "stats.probplot(user_data[target_var], dist=\"norm\", plot=axes[1, 1])\n",
    "axes[1, 1].set_title('Q-Q图(正态性检验)', fontsize=12, fontweight='bold')\n",
    "axes[1, 1].grid(alpha=0.3)\n",
    "\n",
    "# 6. 累积分布图(CDF)\n",
    "sorted_data = np.sort(user_data[target_var])\n",
    "cumulative = np.arange(1, len(sorted_data) + 1) / len(sorted_data)\n",
    "axes[1, 2].plot(sorted_data, cumulative, linewidth=2, color='darkorange')\n",
    "axes[1, 2].axhline(0.5, color='green', linestyle='--', linewidth=1.5, label='中位数位置')\n",
    "axes[1, 2].set_title('累积分布图(CDF)', fontsize=12, fontweight='bold')\n",
    "axes[1, 2].set_xlabel('月消费金额(元)', fontsize=10)\n",
    "axes[1, 2].set_ylabel('累积概率', fontsize=10)\n",
    "axes[1, 2].legend(fontsize=9)\n",
    "axes[1, 2].grid(alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 可视化说明:\")\n",
    "print(\"  - 直方图:显示数据的频数分布\")\n",
    "print(\"  - 核密度图:平滑的概率密度曲线\")\n",
    "print(\"  - 箱线图:显示四分位数和异常值\")\n",
    "print(\"  - 小提琴图:结合箱线图和密度图的优点\")\n",
    "print(\"  - Q-Q图:点越接近直线说明越接近正态分布\")\n",
    "print(\"  - 累积分布图:显示小于等于某值的概率\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 分类型变量分析"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 分析分类变量:会员等级\n",
    "cat_var = '会员等级'\n",
    "\n",
    "print(\"=\"*100)\n",
    "print(f\"📊 分类变量分析:{cat_var}\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 频数统计\n",
    "freq = user_data[cat_var].value_counts()\n",
    "percent = user_data[cat_var].value_counts(normalize=True) * 100\n",
    "\n",
    "stats_df = pd.DataFrame({\n",
    "    '频数': freq,\n",
    "    '占比(%)': percent.round(2)\n",
    "})\n",
    "\n",
    "print(\"\\n【频数与占比】\")\n",
    "print(stats_df)\n",
    "\n",
    "print(\"\\n【关键指标】\")\n",
    "print(f\"  类别数:     {user_data[cat_var].nunique()}\")\n",
    "print(f\"  众数:       {user_data[cat_var].mode()[0]} (出现{freq.max()}次)\")\n",
    "print(f\"  缺失值:     {user_data[cat_var].isnull().sum()}\")\n",
    "print(f\"  最高占比:   {percent.max():.2f}% ({freq.idxmax()})\")\n",
    "print(f\"  最低占比:   {percent.min():.2f}% ({freq.idxmin()})\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 分类变量可视化\n",
    "fig, axes = plt.subplots(1, 3, figsize=(18, 5))\n",
    "\n",
    "# 1. 柱状图\n",
    "level_order = ['普通', '白银', '黄金', '铂金', '钻石']\n",
    "level_counts = user_data['会员等级'].value_counts().reindex(level_order)\n",
    "colors_level = ['gray', 'silver', 'gold', 'lightblue', 'pink']\n",
    "bars = axes[0].bar(level_counts.index, level_counts.values, color=colors_level, edgecolor='black', alpha=0.8)\n",
    "axes[0].set_title('会员等级分布-柱状图', fontsize=13, fontweight='bold')\n",
    "axes[0].set_xlabel('会员等级', fontsize=11)\n",
    "axes[0].set_ylabel('用户数', fontsize=11)\n",
    "axes[0].grid(axis='y', alpha=0.3)\n",
    "# 添加数值标签\n",
    "for i, (bar, count) in enumerate(zip(bars, level_counts.values)):\n",
    "    height = bar.get_height()\n",
    "    axes[0].text(bar.get_x() + bar.get_width()/2., height + 5,\n",
    "                 f'{count}\\n({count/len(user_data)*100:.1f}%)',\n",
    "                 ha='center', va='bottom', fontsize=9)\n",
    "\n",
    "# 2. 饼图\n",
    "axes[1].pie(level_counts.values, labels=level_counts.index, autopct='%1.1f%%',\n",
    "            colors=colors_level, startangle=90, explode=[0.05, 0, 0, 0, 0])\n",
    "axes[1].set_title('会员等级分布-饼图', fontsize=13, fontweight='bold')\n",
    "\n",
    "# 3. 帕累托图(Pareto Chart)\n",
    "sorted_counts = level_counts.sort_values(ascending=False)\n",
    "cumulative_percent = sorted_counts.cumsum() / sorted_counts.sum() * 100\n",
    "\n",
    "ax3_1 = axes[2]\n",
    "ax3_2 = ax3_1.twinx()\n",
    "\n",
    "ax3_1.bar(range(len(sorted_counts)), sorted_counts.values, color='steelblue', alpha=0.7, label='频数')\n",
    "ax3_2.plot(range(len(cumulative_percent)), cumulative_percent.values, color='red', marker='o', linewidth=2, label='累积占比')\n",
    "ax3_2.axhline(80, color='green', linestyle='--', alpha=0.5, label='80%线')\n",
    "\n",
    "ax3_1.set_xlabel('会员等级', fontsize=11)\n",
    "ax3_1.set_ylabel('用户数', fontsize=11, color='steelblue')\n",
    "ax3_2.set_ylabel('累积占比(%)', fontsize=11, color='red')\n",
    "ax3_1.set_xticks(range(len(sorted_counts)))\n",
    "ax3_1.set_xticklabels(sorted_counts.index)\n",
    "ax3_1.set_title('会员等级分布-帕累托图', fontsize=13, fontweight='bold')\n",
    "ax3_1.legend(loc='upper left', fontsize=9)\n",
    "ax3_2.legend(loc='upper right', fontsize=9)\n",
    "ax3_1.grid(axis='y', alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 可视化说明:\")\n",
    "print(\"  - 柱状图:直观显示各类别的频数和占比\")\n",
    "print(\"  - 饼图:显示各类别的比例关系\")\n",
    "print(\"  - 帕累托图:结合频数和累积占比,符合80/20法则\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三、双变量分析(Bivariate Analysis)\n",
    "\n",
    "双变量分析探索**两个变量之间的关系**,根据变量类型分为:\n",
    "- **数值 vs 数值**: 散点图、相关系数\n",
    "- **数值 vs 分类**: 分组箱线图、分组统计\n",
    "- **分类 vs 分类**: 交叉表、堆叠柱状图\n",
    "\n",
    "### 3.1 数值 vs 数值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 分析:月消费金额 vs 月购买次数\n",
    "print(\"=\"*100)\n",
    "print(\"📊 双变量分析:月消费金额 vs 月购买次数\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 计算相关系数\n",
    "corr_pearson = user_data['月消费金额'].corr(user_data['月购买次数'])\n",
    "corr_spearman = user_data['月消费金额'].corr(user_data['月购买次数'], method='spearman')\n",
    "\n",
    "print(\"\\n【相关性分析】\")\n",
    "print(f\"  皮尔逊相关系数(Pearson):    {corr_pearson:.4f}\")\n",
    "print(f\"  斯皮尔曼相关系数(Spearman):  {corr_spearman:.4f}\")\n",
    "\n",
    "if abs(corr_pearson) >= 0.7:\n",
    "    strength = \"强相关\"\n",
    "elif abs(corr_pearson) >= 0.3:\n",
    "    strength = \"中等相关\"\n",
    "else:\n",
    "    strength = \"弱相关\"\n",
    "\n",
    "direction = \"正相关\" if corr_pearson > 0 else \"负相关\"\n",
    "print(f\"\\n  结论: {strength},{direction}\")\n",
    "print(f\"  解读: 月购买次数每增加1次,月消费金额{'增加' if corr_pearson > 0 else '减少'}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数值vs数值可视化\n",
    "fig, axes = plt.subplots(2, 2, figsize=(15, 12))\n",
    "\n",
    "# 1. 散点图\n",
    "axes[0, 0].scatter(user_data['月购买次数'], user_data['月消费金额'], alpha=0.5, s=30, color='steelblue')\n",
    "axes[0, 0].set_title(f'散点图 (相关系数={corr_pearson:.3f})', fontsize=12, fontweight='bold')\n",
    "axes[0, 0].set_xlabel('月购买次数', fontsize=11)\n",
    "axes[0, 0].set_ylabel('月消费金额(元)', fontsize=11)\n",
    "axes[0, 0].grid(alpha=0.3)\n",
    "\n",
    "# 添加回归线\n",
    "z = np.polyfit(user_data['月购买次数'], user_data['月消费金额'], 1)\n",
    "p = np.poly1d(z)\n",
    "axes[0, 0].plot(user_data['月购买次数'], p(user_data['月购买次数']), \"r--\", linewidth=2, label='趋势线')\n",
    "axes[0, 0].legend(fontsize=10)\n",
    "\n",
    "# 2. 六边形热力图(Hexbin)\n",
    "hb = axes[0, 1].hexbin(user_data['月购买次数'], user_data['月消费金额'], gridsize=25, cmap='YlOrRd', mincnt=1)\n",
    "axes[0, 1].set_title('六边形密度图', fontsize=12, fontweight='bold')\n",
    "axes[0, 1].set_xlabel('月购买次数', fontsize=11)\n",
    "axes[0, 1].set_ylabel('月消费金额(元)', fontsize=11)\n",
    "plt.colorbar(hb, ax=axes[0, 1], label='点数量')\n",
    "\n",
    "# 3. 联合分布图(Joint Plot风格)\n",
    "# 主散点图\n",
    "axes[1, 0].scatter(user_data['月购买次数'], user_data['月消费金额'], alpha=0.4, s=20, color='purple')\n",
    "axes[1, 0].set_title('散点图 + 边际直方图', fontsize=12, fontweight='bold')\n",
    "axes[1, 0].set_xlabel('月购买次数', fontsize=11)\n",
    "axes[1, 0].set_ylabel('月消费金额(元)', fontsize=11)\n",
    "axes[1, 0].grid(alpha=0.3)\n",
    "\n",
    "# 4. 分箱后的关系(更清晰)\n",
    "# 将月购买次数分箱\n",
    "user_data['购买次数分组'] = pd.cut(user_data['月购买次数'], bins=[0, 3, 6, 10, 100], labels=['低频(0-3)', '中频(4-6)', '高频(7-10)', '超高频(10+)'])\n",
    "grouped_mean = user_data.groupby('购买次数分组', observed=True)['月消费金额'].mean()\n",
    "grouped_std = user_data.groupby('购买次数分组', observed=True)['月消费金额'].std()\n",
    "\n",
    "x_pos = range(len(grouped_mean))\n",
    "axes[1, 1].bar(x_pos, grouped_mean.values, yerr=grouped_std.values, capsize=5, color='lightcoral', alpha=0.7, edgecolor='black')\n",
    "axes[1, 1].set_title('分组后的平均消费(含标准差)', fontsize=12, fontweight='bold')\n",
    "axes[1, 1].set_xlabel('购买频次分组', fontsize=11)\n",
    "axes[1, 1].set_ylabel('平均月消费金额(元)', fontsize=11)\n",
    "axes[1, 1].set_xticks(x_pos)\n",
    "axes[1, 1].set_xticklabels(grouped_mean.index, rotation=15)\n",
    "axes[1, 1].grid(axis='y', alpha=0.3)\n",
    "# 添加数值标签\n",
    "for i, v in enumerate(grouped_mean.values):\n",
    "    axes[1, 1].text(i, v + grouped_std.values[i] + 100, f'{v:.0f}', ha='center', fontsize=9)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 可视化说明:\")\n",
    "print(\"  - 散点图:最直观的双变量关系展示\")\n",
    "print(\"  - 六边形密度图:适用于大量数据点,显示密集区域\")\n",
    "print(\"  - 分组柱状图:将连续变量分箱,更清晰展示趋势\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 数值 vs 分类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 分析:月消费金额 vs 会员等级\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"📊 双变量分析:月消费金额 vs 会员等级\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 分组统计\n",
    "level_stats = user_data.groupby('会员等级')['月消费金额'].agg([\n",
    "    ('数量', 'count'),\n",
    "    ('均值', 'mean'),\n",
    "    ('中位数', 'median'),\n",
    "    ('标准差', 'std'),\n",
    "    ('最小值', 'min'),\n",
    "    ('最大值', 'max')\n",
    "]).reindex(level_order).round(2)\n",
    "\n",
    "print(\"\\n【分组统计】\")\n",
    "print(level_stats)\n",
    "\n",
    "# 方差分析(ANOVA)\n",
    "groups = [user_data[user_data['会员等级']==level]['月消费金额'].dropna() for level in level_order]\n",
    "f_stat, p_value = stats.f_oneway(*groups)\n",
    "\n",
    "print(\"\\n【方差分析(ANOVA)】\")\n",
    "print(f\"  F统计量: {f_stat:.4f}\")\n",
    "print(f\"  p值:     {p_value:.6f}\")\n",
    "if p_value < 0.05:\n",
    "    print(\"  ✅ 结论: 不同会员等级的月消费金额存在显著差异(p<0.05)\")\n",
    "else:\n",
    "    print(\"  ❌ 结论: 不同会员等级的月消费金额无显著差异(p≥0.05)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数值vs分类可视化\n",
    "fig, axes = plt.subplots(2, 2, figsize=(16, 12))\n",
    "\n",
    "# 1. 分组箱线图\n",
    "data_by_level = [user_data[user_data['会员等级']==level]['月消费金额'].dropna() for level in level_order]\n",
    "bp1 = axes[0, 0].boxplot(data_by_level, labels=level_order, patch_artist=True,\n",
    "                          boxprops=dict(facecolor='lightblue', alpha=0.7),\n",
    "                          medianprops=dict(color='red', linewidth=2))\n",
    "axes[0, 0].set_title('各会员等级月消费金额箱线图', fontsize=13, fontweight='bold')\n",
    "axes[0, 0].set_xlabel('会员等级', fontsize=11)\n",
    "axes[0, 0].set_ylabel('月消费金额(元)', fontsize=11)\n",
    "axes[0, 0].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 2. 分组小提琴图\n",
    "positions = range(len(level_order))\n",
    "parts = axes[0, 1].violinplot(data_by_level, positions=positions, widths=0.7, showmeans=True, showmedians=True)\n",
    "axes[0, 1].set_title('各会员等级月消费金额小提琴图', fontsize=13, fontweight='bold')\n",
    "axes[0, 1].set_xlabel('会员等级', fontsize=11)\n",
    "axes[0, 1].set_ylabel('月消费金额(元)', fontsize=11)\n",
    "axes[0, 1].set_xticks(positions)\n",
    "axes[0, 1].set_xticklabels(level_order)\n",
    "axes[0, 1].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 3. 均值对比柱状图\n",
    "mean_by_level = user_data.groupby('会员等级')['月消费金额'].mean().reindex(level_order)\n",
    "bars = axes[1, 0].bar(level_order, mean_by_level.values, color=colors_level, edgecolor='black', alpha=0.8)\n",
    "axes[1, 0].set_title('各会员等级平均月消费金额', fontsize=13, fontweight='bold')\n",
    "axes[1, 0].set_xlabel('会员等级', fontsize=11)\n",
    "axes[1, 0].set_ylabel('平均月消费金额(元)', fontsize=11)\n",
    "axes[1, 0].grid(axis='y', alpha=0.3)\n",
    "# 添加数值标签\n",
    "for bar, val in zip(bars, mean_by_level.values):\n",
    "    height = bar.get_height()\n",
    "    axes[1, 0].text(bar.get_x() + bar.get_width()/2., height + 50,\n",
    "                    f'{val:.0f}', ha='center', va='bottom', fontsize=10)\n",
    "\n",
    "# 4. 分组直方图(重叠)\n",
    "for level, color in zip(level_order, colors_level):\n",
    "    level_data = user_data[user_data['会员等级']==level]['月消费金额']\n",
    "    axes[1, 1].hist(level_data, bins=30, alpha=0.5, label=level, color=color, edgecolor='black')\n",
    "axes[1, 1].set_title('各会员等级月消费分布(重叠)', fontsize=13, fontweight='bold')\n",
    "axes[1, 1].set_xlabel('月消费金额(元)', fontsize=11)\n",
    "axes[1, 1].set_ylabel('频数', fontsize=11)\n",
    "axes[1, 1].legend(fontsize=9)\n",
    "axes[1, 1].grid(axis='y', alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 业务洞察:\")\n",
    "print(f\"  - 钻石会员平均消费({mean_by_level['钻石']:.0f}元)是普通会员({mean_by_level['普通']:.0f}元)的{mean_by_level['钻石']/mean_by_level['普通']:.1f}倍\")\n",
    "print(f\"  - 会员等级越高,消费金额中位数和均值差距越大,说明高等级会员内部差异大\")\n",
    "print(f\"  - 建议:针对高等级会员提供个性化服务,针对低等级会员设计升级激励\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 分类 vs 分类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 分析:会员等级 vs 城市等级\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"📊 双变量分析:会员等级 vs 城市等级\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 交叉表\n",
    "crosstab = pd.crosstab(user_data['会员等级'], user_data['城市等级'], margins=True)\n",
    "crosstab = crosstab.reindex(level_order + ['All'], fill_value=0)\n",
    "crosstab = crosstab[['一线', '二线', '三线', '四线', 'All']]\n",
    "\n",
    "print(\"\\n【交叉表-频数】\")\n",
    "print(crosstab)\n",
    "\n",
    "# 百分比交叉表(按行)\n",
    "crosstab_pct = pd.crosstab(user_data['会员等级'], user_data['城市等级'], normalize='index') * 100\n",
    "crosstab_pct = crosstab_pct.reindex(level_order, fill_value=0)\n",
    "crosstab_pct = crosstab_pct[['一线', '二线', '三线', '四线']]\n",
    "\n",
    "print(\"\\n【交叉表-行百分比(%)】\")\n",
    "print(crosstab_pct.round(2))\n",
    "\n",
    "# 卡方检验\n",
    "crosstab_test = pd.crosstab(user_data['会员等级'], user_data['城市等级'])\n",
    "chi2, p_value, dof, expected = stats.chi2_contingency(crosstab_test)\n",
    "\n",
    "print(\"\\n【卡方检验(Chi-square Test)】\")\n",
    "print(f\"  卡方统计量: {chi2:.4f}\")\n",
    "print(f\"  p值:        {p_value:.6f}\")\n",
    "print(f\"  自由度:     {dof}\")\n",
    "if p_value < 0.05:\n",
    "    print(\"  ✅ 结论: 会员等级与城市等级存在显著关联(p<0.05)\")\n",
    "else:\n",
    "    print(\"  ❌ 结论: 会员等级与城市等级无显著关联(p≥0.05)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 分类vs分类可视化\n",
    "fig, axes = plt.subplots(2, 2, figsize=(16, 12))\n",
    "\n",
    "# 1. 堆叠柱状图(数量)\n",
    "crosstab_plot = pd.crosstab(user_data['会员等级'], user_data['城市等级'])\n",
    "crosstab_plot = crosstab_plot.reindex(level_order)[['一线', '二线', '三线', '四线']]\n",
    "crosstab_plot.plot(kind='bar', stacked=True, ax=axes[0, 0], color=colors_city, edgecolor='black', alpha=0.8)\n",
    "axes[0, 0].set_title('会员等级 vs 城市等级分布(堆叠)', fontsize=13, fontweight='bold')\n",
    "axes[0, 0].set_xlabel('会员等级', fontsize=11)\n",
    "axes[0, 0].set_ylabel('用户数', fontsize=11)\n",
    "axes[0, 0].legend(title='城市等级', fontsize=9)\n",
    "axes[0, 0].tick_params(axis='x', rotation=0)\n",
    "axes[0, 0].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 2. 堆叠柱状图(百分比)\n",
    "crosstab_pct.plot(kind='bar', stacked=True, ax=axes[0, 1], color=colors_city, edgecolor='black', alpha=0.8)\n",
    "axes[0, 1].set_title('会员等级 vs 城市等级分布(百分比堆叠)', fontsize=13, fontweight='bold')\n",
    "axes[0, 1].set_xlabel('会员等级', fontsize=11)\n",
    "axes[0, 1].set_ylabel('占比(%)', fontsize=11)\n",
    "axes[0, 1].legend(title='城市等级', fontsize=9)\n",
    "axes[0, 1].tick_params(axis='x', rotation=0)\n",
    "axes[0, 1].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 3. 分组柱状图(并排)\n",
    "crosstab_plot.plot(kind='bar', ax=axes[1, 0], color=colors_city, edgecolor='black', alpha=0.8, width=0.8)\n",
    "axes[1, 0].set_title('会员等级 vs 城市等级分布(并排)', fontsize=13, fontweight='bold')\n",
    "axes[1, 0].set_xlabel('会员等级', fontsize=11)\n",
    "axes[1, 0].set_ylabel('用户数', fontsize=11)\n",
    "axes[1, 0].legend(title='城市等级', fontsize=9)\n",
    "axes[1, 0].tick_params(axis='x', rotation=0)\n",
    "axes[1, 0].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 4. 热力图\n",
    "sns.heatmap(crosstab_plot, annot=True, fmt='d', cmap='Blues', ax=axes[1, 1], \n",
    "            cbar_kws={'label': '用户数'}, linewidths=1, linecolor='white')\n",
    "axes[1, 1].set_title('会员等级 vs 城市等级热力图', fontsize=13, fontweight='bold')\n",
    "axes[1, 1].set_xlabel('城市等级', fontsize=11)\n",
    "axes[1, 1].set_ylabel('会员等级', fontsize=11)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 可视化说明:\")\n",
    "print(\"  - 堆叠柱状图:显示各分类的组成结构\")\n",
    "print(\"  - 并排柱状图:便于比较各类别在不同组的分布\")\n",
    "print(\"  - 热力图:颜色深浅直观显示数量多少\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 四、多变量分析(Multivariate Analysis)\n",
    "\n",
    "多变量分析同时考虑**三个或更多变量之间的关系**。\n",
    "\n",
    "### 4.1 相关系数矩阵"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 选择关键数值变量\n",
    "numeric_features = ['年龄', '注册天数', '月购买次数', '月消费金额', '月登录次数', '平均浏览时长', '客单价']\n",
    "\n",
    "# 计算相关系数矩阵\n",
    "corr_matrix = user_data[numeric_features].corr()\n",
    "\n",
    "print(\"=\"*100)\n",
    "print(\"📊 多变量相关性分析\")\n",
    "print(\"=\"*100)\n",
    "print(\"\\n【相关系数矩阵】\")\n",
    "print(corr_matrix.round(3))\n",
    "\n",
    "# 找出强相关的变量对\n",
    "print(\"\\n【强相关变量对(|r|≥0.5)】\")\n",
    "strong_corr = []\n",
    "for i in range(len(corr_matrix.columns)):\n",
    "    for j in range(i+1, len(corr_matrix.columns)):\n",
    "        corr_val = corr_matrix.iloc[i, j]\n",
    "        if abs(corr_val) >= 0.5:\n",
    "            var1 = corr_matrix.columns[i]\n",
    "            var2 = corr_matrix.columns[j]\n",
    "            strong_corr.append((var1, var2, corr_val))\n",
    "\n",
    "if strong_corr:\n",
    "    for var1, var2, corr_val in sorted(strong_corr, key=lambda x: abs(x[2]), reverse=True):\n",
    "        direction = \"正相关\" if corr_val > 0 else \"负相关\"\n",
    "        print(f\"  {var1} <-> {var2}: r={corr_val:.3f} ({direction})\")\n",
    "else:\n",
    "    print(\"  未发现强相关变量对\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 相关系数矩阵可视化\n",
    "fig, axes = plt.subplots(1, 2, figsize=(18, 7))\n",
    "\n",
    "# 1. 热力图\n",
    "sns.heatmap(corr_matrix, annot=True, fmt='.2f', cmap='coolwarm', center=0,\n",
    "            square=True, linewidths=1, cbar_kws={'label': '相关系数'},\n",
    "            vmin=-1, vmax=1, ax=axes[0])\n",
    "axes[0].set_title('变量相关系数热力图', fontsize=14, fontweight='bold')\n",
    "\n",
    "# 2. 只显示上三角(去除重复)\n",
    "mask = np.triu(np.ones_like(corr_matrix, dtype=bool))\n",
    "sns.heatmap(corr_matrix, mask=mask, annot=True, fmt='.2f', cmap='coolwarm', center=0,\n",
    "            square=True, linewidths=1, cbar_kws={'label': '相关系数'},\n",
    "            vmin=-1, vmax=1, ax=axes[1])\n",
    "axes[1].set_title('变量相关系数热力图(上三角)', fontsize=14, fontweight='bold')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 图表说明:\")\n",
    "print(\"  - 红色表示正相关,蓝色表示负相关\")\n",
    "print(\"  - 颜色越深表示相关性越强\")\n",
    "print(\"  - 对角线为1(自己与自己完全相关)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 散点图矩阵"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 选择4个关键变量绘制散点图矩阵\n",
    "key_features = ['月消费金额', '月购买次数', '月登录次数', '客单价']\n",
    "\n",
    "# 创建散点图矩阵\n",
    "from pandas.plotting import scatter_matrix\n",
    "\n",
    "fig = plt.figure(figsize=(14, 14))\n",
    "scatter_matrix(user_data[key_features], alpha=0.3, figsize=(14, 14), diagonal='hist',\n",
    "               c=user_data['客户评分'], cmap='viridis', s=20, edgecolors='black', linewidth=0.5)\n",
    "plt.suptitle('关键变量散点图矩阵(颜色=客户评分)', fontsize=15, fontweight='bold', y=0.995)\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 图表说明:\")\n",
    "print(\"  - 对角线:各变量的分布直方图\")\n",
    "print(\"  - 非对角线:两两变量的散点图\")\n",
    "print(\"  - 颜色:客户评分(黄色=高分,紫色=低分)\")\n",
    "print(\"  - 作用:快速识别变量间的线性/非线性关系\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 多维度分析:添加第三变量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 分析:月消费金额 vs 月购买次数,按会员等级分组\n",
    "fig, axes = plt.subplots(1, 2, figsize=(16, 6))\n",
    "\n",
    "# 1. 按会员等级着色的散点图\n",
    "for level, color in zip(level_order, colors_level):\n",
    "    level_data = user_data[user_data['会员等级']==level]\n",
    "    axes[0].scatter(level_data['月购买次数'], level_data['月消费金额'], \n",
    "                    alpha=0.6, s=40, label=level, color=color, edgecolors='black', linewidth=0.5)\n",
    "axes[0].set_title('月消费金额 vs 月购买次数(按会员等级)', fontsize=13, fontweight='bold')\n",
    "axes[0].set_xlabel('月购买次数', fontsize=11)\n",
    "axes[0].set_ylabel('月消费金额(元)', fontsize=11)\n",
    "axes[0].legend(fontsize=10, loc='upper left')\n",
    "axes[0].grid(alpha=0.3)\n",
    "\n",
    "# 2. 气泡图(气泡大小=客单价)\n",
    "scatter = axes[1].scatter(user_data['月购买次数'], user_data['月消费金额'],\n",
    "                          s=user_data['客单价']/10,  # 缩小以便显示\n",
    "                          c=user_data['客户评分'], cmap='RdYlGn',\n",
    "                          alpha=0.5, edgecolors='black', linewidth=0.5)\n",
    "axes[1].set_title('月消费金额 vs 月购买次数(气泡=客单价,颜色=评分)', fontsize=13, fontweight='bold')\n",
    "axes[1].set_xlabel('月购买次数', fontsize=11)\n",
    "axes[1].set_ylabel('月消费金额(元)', fontsize=11)\n",
    "cbar = plt.colorbar(scatter, ax=axes[1], label='客户评分')\n",
    "axes[1].grid(alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 多维度洞察:\")\n",
    "print(\"  - 不同会员等级的消费模式明显不同\")\n",
    "print(\"  - 钻石和铂金会员:低频高额(客单价高)\")\n",
    "print(\"  - 普通和白银会员:高频低额(客单价低)\")\n",
    "print(\"  - 气泡图显示:高消费、高频次、高客单价的用户评分也较高\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 五、完整EDA项目实战\n",
    "\n",
    "现在运用所有学到的方法,完成一个完整的EDA报告。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建完整的EDA报告函数\n",
    "def generate_eda_report(df, title=\"数据探索性分析报告\"):\n",
    "    \"\"\"\n",
    "    生成完整的EDA报告\n",
    "    \"\"\"\n",
    "    print(\"=\"*100)\n",
    "    print(f\"📊 {title}\")\n",
    "    print(\"=\"*100)\n",
    "    \n",
    "    # 1. 数据概览\n",
    "    print(\"\\n【一、数据概览】\")\n",
    "    print(\"-\"*100)\n",
    "    print(f\"数据集规模: {df.shape[0]}行 × {df.shape[1]}列\")\n",
    "    print(f\"数值型字段: {len(df.select_dtypes(include=[np.number]).columns)}个\")\n",
    "    print(f\"分类型字段: {len(df.select_dtypes(include=['object']).columns)}个\")\n",
    "    print(f\"总缺失值:   {df.isnull().sum().sum()}个 ({df.isnull().sum().sum()/(df.shape[0]*df.shape[1])*100:.2f}%)\")\n",
    "    print(f\"总重复行:   {df.duplicated().sum()}行\")\n",
    "    \n",
    "    # 2. 缺失值分析\n",
    "    print(\"\\n【二、缺失值分析】\")\n",
    "    print(\"-\"*100)\n",
    "    missing = df.isnull().sum()\n",
    "    missing = missing[missing > 0].sort_values(ascending=False)\n",
    "    if len(missing) > 0:\n",
    "        for col, count in missing.items():\n",
    "            pct = count / len(df) * 100\n",
    "            print(f\"{col}: {count}个缺失 ({pct:.2f}%)\")\n",
    "    else:\n",
    "        print(\"✅ 无缺失值\")\n",
    "    \n",
    "    # 3. 数值型变量统计\n",
    "    print(\"\\n【三、数值型变量统计摘要】\")\n",
    "    print(\"-\"*100)\n",
    "    numeric_cols = df.select_dtypes(include=[np.number]).columns\n",
    "    if len(numeric_cols) > 0:\n",
    "        desc = df[numeric_cols].describe().T\n",
    "        desc['偏度'] = df[numeric_cols].skew()\n",
    "        desc['峰度'] = df[numeric_cols].kurt()\n",
    "        print(desc.round(2))\n",
    "    \n",
    "    # 4. 分类型变量统计\n",
    "    print(\"\\n【四、分类型变量统计摘要】\")\n",
    "    print(\"-\"*100)\n",
    "    cat_cols = df.select_dtypes(include=['object']).columns\n",
    "    if len(cat_cols) > 0:\n",
    "        for col in cat_cols:\n",
    "            print(f\"\\n{col}:\")\n",
    "            value_counts = df[col].value_counts()\n",
    "            for val, count in value_counts.head(5).items():\n",
    "                print(f\"  {val}: {count}个 ({count/len(df)*100:.2f}%)\")\n",
    "    \n",
    "    # 5. 异常值检测\n",
    "    print(\"\\n【五、异常值检测(IQR方法)】\")\n",
    "    print(\"-\"*100)\n",
    "    for col in numeric_cols:\n",
    "        Q1 = df[col].quantile(0.25)\n",
    "        Q3 = df[col].quantile(0.75)\n",
    "        IQR = Q3 - Q1\n",
    "        lower = Q1 - 1.5 * IQR\n",
    "        upper = Q3 + 1.5 * IQR\n",
    "        outliers = df[(df[col] < lower) | (df[col] > upper)]\n",
    "        if len(outliers) > 0:\n",
    "            print(f\"{col}: {len(outliers)}个异常值 ({len(outliers)/len(df)*100:.2f}%)\")\n",
    "    \n",
    "    print(\"\\n\" + \"=\"*100)\n",
    "    print(\"✅ EDA报告生成完成!\")\n",
    "    print(\"=\"*100)\n",
    "\n",
    "# 调用函数生成报告\n",
    "generate_eda_report(user_data, \"电商用户行为数据EDA报告\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 生成完整的可视化报告\n",
    "fig = plt.figure(figsize=(20, 16))\n",
    "gs = fig.add_gridspec(4, 4, hspace=0.3, wspace=0.3)\n",
    "\n",
    "# 第一行:单变量分析(数值型)\n",
    "ax1 = fig.add_subplot(gs[0, 0])\n",
    "ax1.hist(user_data['年龄'], bins=30, color='skyblue', edgecolor='black', alpha=0.7)\n",
    "ax1.set_title('年龄分布', fontsize=11, fontweight='bold')\n",
    "ax1.set_xlabel('年龄', fontsize=9)\n",
    "ax1.grid(axis='y', alpha=0.3)\n",
    "\n",
    "ax2 = fig.add_subplot(gs[0, 1])\n",
    "ax2.hist(user_data['月消费金额'], bins=40, color='coral', edgecolor='black', alpha=0.7)\n",
    "ax2.set_title('月消费金额分布', fontsize=11, fontweight='bold')\n",
    "ax2.set_xlabel('金额(元)', fontsize=9)\n",
    "ax2.grid(axis='y', alpha=0.3)\n",
    "\n",
    "ax3 = fig.add_subplot(gs[0, 2])\n",
    "level_counts = user_data['会员等级'].value_counts().reindex(level_order)\n",
    "ax3.bar(level_counts.index, level_counts.values, color=colors_level, edgecolor='black', alpha=0.8)\n",
    "ax3.set_title('会员等级分布', fontsize=11, fontweight='bold')\n",
    "ax3.tick_params(axis='x', rotation=45, labelsize=8)\n",
    "ax3.grid(axis='y', alpha=0.3)\n",
    "\n",
    "ax4 = fig.add_subplot(gs[0, 3])\n",
    "city_counts = user_data['城市等级'].value_counts().reindex(['一线', '二线', '三线', '四线'])\n",
    "ax4.pie(city_counts.values, labels=city_counts.index, autopct='%1.1f%%', colors=colors_city, startangle=90)\n",
    "ax4.set_title('城市等级分布', fontsize=11, fontweight='bold')\n",
    "\n",
    "# 第二行:数值vs数值\n",
    "ax5 = fig.add_subplot(gs[1, :2])\n",
    "for level, color in zip(level_order, colors_level):\n",
    "    level_data = user_data[user_data['会员等级']==level]\n",
    "    ax5.scatter(level_data['月购买次数'], level_data['月消费金额'], \n",
    "                alpha=0.5, s=30, label=level, color=color)\n",
    "ax5.set_title('月消费金额 vs 月购买次数(按会员等级)', fontsize=11, fontweight='bold')\n",
    "ax5.set_xlabel('月购买次数', fontsize=9)\n",
    "ax5.set_ylabel('月消费金额(元)', fontsize=9)\n",
    "ax5.legend(fontsize=8, loc='upper left')\n",
    "ax5.grid(alpha=0.3)\n",
    "\n",
    "ax6 = fig.add_subplot(gs[1, 2:])\n",
    "data_by_level = [user_data[user_data['会员等级']==level]['月消费金额'].dropna() for level in level_order]\n",
    "ax6.boxplot(data_by_level, labels=level_order, patch_artist=True,\n",
    "            boxprops=dict(facecolor='lightblue', alpha=0.7),\n",
    "            medianprops=dict(color='red', linewidth=2))\n",
    "ax6.set_title('各会员等级月消费金额箱线图', fontsize=11, fontweight='bold')\n",
    "ax6.tick_params(axis='x', labelsize=8)\n",
    "ax6.grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 第三行:相关性分析\n",
    "ax7 = fig.add_subplot(gs[2, :])\n",
    "sns.heatmap(corr_matrix, annot=True, fmt='.2f', cmap='coolwarm', center=0,\n",
    "            square=True, linewidths=1, cbar_kws={'label': '相关系数'},\n",
    "            vmin=-1, vmax=1, ax=ax7)\n",
    "ax7.set_title('变量相关系数热力图', fontsize=12, fontweight='bold')\n",
    "\n",
    "# 第四行:分类vs分类\n",
    "ax8 = fig.add_subplot(gs[3, :2])\n",
    "crosstab_plot = pd.crosstab(user_data['会员等级'], user_data['城市等级'])\n",
    "crosstab_plot = crosstab_plot.reindex(level_order)[['一线', '二线', '三线', '四线']]\n",
    "crosstab_plot.plot(kind='bar', stacked=True, ax=ax8, color=colors_city, edgecolor='black', alpha=0.8)\n",
    "ax8.set_title('会员等级 vs 城市等级分布', fontsize=11, fontweight='bold')\n",
    "ax8.tick_params(axis='x', rotation=0, labelsize=8)\n",
    "ax8.legend(title='城市等级', fontsize=8)\n",
    "ax8.grid(axis='y', alpha=0.3)\n",
    "\n",
    "ax9 = fig.add_subplot(gs[3, 2:])\n",
    "repurchase_by_level = pd.crosstab(user_data['会员等级'], user_data['是否复购'], normalize='index') * 100\n",
    "repurchase_by_level = repurchase_by_level.reindex(level_order)\n",
    "repurchase_by_level.plot(kind='bar', ax=ax9, color=['red', 'green'], edgecolor='black', alpha=0.8)\n",
    "ax9.set_title('各会员等级复购率', fontsize=11, fontweight='bold')\n",
    "ax9.set_ylabel('占比(%)', fontsize=9)\n",
    "ax9.tick_params(axis='x', rotation=0, labelsize=8)\n",
    "ax9.legend(title='是否复购', fontsize=8)\n",
    "ax9.grid(axis='y', alpha=0.3)\n",
    "\n",
    "plt.suptitle('电商用户行为数据完整EDA可视化报告', fontsize=16, fontweight='bold', y=0.998)\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n✅ 完整可视化报告生成完成!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 六、EDA最佳实践与技巧\n",
    "\n",
    "### 6.1 EDA流程清单\n",
    "\n",
    "```\n",
    "✅ 第一步:数据加载与概览\n",
    "   - df.head(), df.tail(), df.sample()\n",
    "   - df.info(), df.shape\n",
    "   - df.columns, df.dtypes\n",
    "\n",
    "✅ 第二步:数据质量检查\n",
    "   - df.isnull().sum()  # 缺失值\n",
    "   - df.duplicated().sum()  # 重复值\n",
    "   - df.describe()  # 统计摘要\n",
    "\n",
    "✅ 第三步:单变量分析\n",
    "   - 数值型:直方图、箱线图、统计指标\n",
    "   - 分类型:频数表、柱状图、饼图\n",
    "\n",
    "✅ 第四步:双变量分析\n",
    "   - 数值vs数值:散点图、相关系数\n",
    "   - 数值vs分类:分组箱线图、ANOVA\n",
    "   - 分类vs分类:交叉表、卡方检验\n",
    "\n",
    "✅ 第五步:多变量分析\n",
    "   - 相关系数矩阵热力图\n",
    "   - 散点图矩阵\n",
    "   - 多维度可视化(颜色、大小、形状)\n",
    "\n",
    "✅ 第六步:异常检测\n",
    "   - IQR方法\n",
    "   - Z-score方法\n",
    "   - 业务规则\n",
    "\n",
    "✅ 第七步:总结洞察\n",
    "   - 数据特征总结\n",
    "   - 关键发现列举\n",
    "   - 业务建议提出\n",
    "```\n",
    "\n",
    "### 6.2 常用代码模板"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# EDA常用代码模板\n",
    "\n",
    "# 1. 快速数据概览\n",
    "def quick_overview(df):\n",
    "    print(f\"数据维度: {df.shape}\")\n",
    "    print(f\"\\n数据类型:\\n{df.dtypes}\")\n",
    "    print(f\"\\n缺失值:\\n{df.isnull().sum()}\")\n",
    "    print(f\"\\n数值型统计:\\n{df.describe().T}\")\n",
    "    return df.head()\n",
    "\n",
    "# 2. 批量绘制直方图\n",
    "def plot_histograms(df, numeric_cols, ncols=3):\n",
    "    nrows = (len(numeric_cols) + ncols - 1) // ncols\n",
    "    fig, axes = plt.subplots(nrows, ncols, figsize=(ncols*5, nrows*4))\n",
    "    axes = axes.flatten() if nrows > 1 else [axes]\n",
    "    \n",
    "    for idx, col in enumerate(numeric_cols):\n",
    "        axes[idx].hist(df[col].dropna(), bins=30, color='steelblue', edgecolor='black', alpha=0.7)\n",
    "        axes[idx].set_title(f'{col}分布', fontweight='bold')\n",
    "        axes[idx].set_xlabel(col)\n",
    "        axes[idx].grid(axis='y', alpha=0.3)\n",
    "    \n",
    "    # 隐藏多余的子图\n",
    "    for idx in range(len(numeric_cols), len(axes)):\n",
    "        axes[idx].axis('off')\n",
    "    \n",
    "    plt.tight_layout()\n",
    "    plt.show()\n",
    "\n",
    "# 3. 批量绘制箱线图\n",
    "def plot_boxplots(df, numeric_cols, ncols=3):\n",
    "    nrows = (len(numeric_cols) + ncols - 1) // ncols\n",
    "    fig, axes = plt.subplots(nrows, ncols, figsize=(ncols*5, nrows*4))\n",
    "    axes = axes.flatten() if nrows > 1 else [axes]\n",
    "    \n",
    "    for idx, col in enumerate(numeric_cols):\n",
    "        axes[idx].boxplot(df[col].dropna(), patch_artist=True,\n",
    "                          boxprops=dict(facecolor='lightblue'))\n",
    "        axes[idx].set_title(f'{col}箱线图', fontweight='bold')\n",
    "        axes[idx].set_ylabel(col)\n",
    "        axes[idx].grid(axis='y', alpha=0.3)\n",
    "    \n",
    "    for idx in range(len(numeric_cols), len(axes)):\n",
    "        axes[idx].axis('off')\n",
    "    \n",
    "    plt.tight_layout()\n",
    "    plt.show()\n",
    "\n",
    "print(\"✅ EDA常用函数定义完成!\")\n",
    "print(\"使用方法:\")\n",
    "print(\"  quick_overview(user_data)\")\n",
    "print(\"  plot_histograms(user_data, numeric_features)\")\n",
    "print(\"  plot_boxplots(user_data, numeric_features)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 七、课程总结\n",
    "\n",
    "### 核心知识点\n",
    "\n",
    "1. **EDA六步流程**\n",
    "   - 数据概览 → 单变量分析 → 双变量分析 → 多变量分析 → 异常检测 → 总结洞察\n",
    "\n",
    "2. **单变量分析**\n",
    "   - 数值型:直方图、KDE、箱线图、统计指标(均值、中位数、标准差、偏度、峰度)\n",
    "   - 分类型:频数表、柱状图、饼图、帕累托图\n",
    "\n",
    "3. **双变量分析**\n",
    "   - 数值vs数值:散点图、相关系数、回归线\n",
    "   - 数值vs分类:分组箱线图、小提琴图、ANOVA检验\n",
    "   - 分类vs分类:交叉表、堆叠柱状图、热力图、卡方检验\n",
    "\n",
    "4. **多变量分析**\n",
    "   - 相关系数矩阵热力图\n",
    "   - 散点图矩阵\n",
    "   - 多维度可视化(颜色、大小、形状编码)\n",
    "\n",
    "5. **关键Pandas/可视化函数**\n",
    "   ```python\n",
    "   # 数据概览\n",
    "   df.info(), df.describe(), df.head()\n",
    "   \n",
    "   # 统计分析\n",
    "   df['col'].mean(), median(), std(), skew(), kurt()\n",
    "   df.corr()  # 相关系数矩阵\n",
    "   pd.crosstab()  # 交叉表\n",
    "   \n",
    "   # 可视化\n",
    "   plt.hist(), plt.boxplot(), plt.scatter()\n",
    "   sns.heatmap(), sns.violinplot(), sns.pairplot()\n",
    "   ```\n",
    "\n",
    "### 业务应用场景\n",
    "\n",
    "- **用户分析**: 探索用户特征、行为模式、细分群体\n",
    "- **销售分析**: 发现销售趋势、影响因素、异常订单\n",
    "- **产品分析**: 分析产品表现、用户偏好、改进方向\n",
    "- **风险分析**: 识别异常交易、欺诈模式\n",
    "\n",
    "### EDA vs 传统报表\n",
    "\n",
    "| 特点 | 传统报表 | EDA |\n",
    "|------|----------|-----|\n",
    "| 目标 | 展示已知指标 | 发现未知规律 |\n",
    "| 方法 | 预定义维度 | 自由探索 |\n",
    "| 产出 | 固定报表 | 洞察和假设 |\n",
    "| 工具 | Excel, BI工具 | Python, R |\n",
    "| 适用 | 日常监控 | 深度分析 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 八、课后作业\n",
    "\n",
    "### 作业1:学生成绩EDA(基础)\n",
    "\n",
    "对某班级学生的考试成绩进行EDA,数据包含学号、性别、班级、数学、语文、英语成绩。要求:\n",
    "1. 生成数据质量报告(缺失值、异常值)\n",
    "2. 单变量分析:各科成绩分布(直方图、箱线图)\n",
    "3. 双变量分析:科目间相关性、性别vs成绩\n",
    "4. 找出成绩异常的学生\n",
    "5. 撰写分析结论"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 作业1数据生成\n",
    "np.random.seed(42)\n",
    "n_students = 150\n",
    "\n",
    "homework1_data = pd.DataFrame({\n",
    "    '学号': [f'S{str(i).zfill(4)}' for i in range(1, n_students+1)],\n",
    "    '性别': np.random.choice(['男', '女'], n_students),\n",
    "    '班级': np.random.choice(['1班', '2班', '3班'], n_students),\n",
    "    '数学': np.random.normal(75, 12, n_students).clip(0, 100).round(1),\n",
    "    '语文': np.random.normal(78, 10, n_students).clip(0, 100).round(1),\n",
    "    '英语': np.random.normal(72, 15, n_students).clip(0, 100).round(1)\n",
    "})\n",
    "\n",
    "# 添加一些缺失值\n",
    "homework1_data.loc[np.random.choice(homework1_data.index, 5), '数学'] = np.nan\n",
    "\n",
    "print(\"作业1数据预览:\")\n",
    "print(homework1_data.head(10))\n",
    "\n",
    "# TODO: 在此完成作业1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 作业2:员工薪资EDA(进阶)\n",
    "\n",
    "对某公司员工薪资数据进行EDA,数据包含部门、职级、工作年限、月薪、绩效等。要求:\n",
    "1. 完整的数据概览报告\n",
    "2. 单变量分析:薪资分布(多种可视化)\n",
    "3. 双变量分析:薪资vs部门、薪资vs职级、薪资vs工作年限\n",
    "4. 多变量分析:相关系数矩阵、散点图矩阵\n",
    "5. 异常值检测和处理建议\n",
    "6. 业务洞察和建议"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 作业2数据生成\n",
    "np.random.seed(42)\n",
    "n_employees = 500\n",
    "\n",
    "homework2_data = pd.DataFrame({\n",
    "    '员工ID': [f'E{str(i).zfill(5)}' for i in range(1, n_employees+1)],\n",
    "    '部门': np.random.choice(['技术', '销售', '市场', '运营'], n_employees, p=[0.4, 0.3, 0.2, 0.1]),\n",
    "    '职级': np.random.choice(['初级', '中级', '高级', '专家'], n_employees, p=[0.4, 0.35, 0.2, 0.05]),\n",
    "    '工作年限': np.random.poisson(lam=3, size=n_employees).clip(0, 20) + 1,\n",
    "    '月薪': np.random.normal(15, 5, n_employees).clip(5, 50),\n",
    "    '绩效评分': np.random.choice([60, 70, 80, 90, 100], n_employees, p=[0.05, 0.15, 0.4, 0.3, 0.1]),\n",
    "    '年终奖': np.random.gamma(2, 3, n_employees).clip(0, 30)\n",
    "})\n",
    "\n",
    "# 根据职级调整薪资\n",
    "level_mult = {'初级': 0.8, '中级': 1.0, '高级': 1.5, '专家': 2.5}\n",
    "homework2_data['月薪'] = homework2_data.apply(lambda row: row['月薪'] * level_mult[row['职级']], axis=1).round(2)\n",
    "\n",
    "print(\"作业2数据预览:\")\n",
    "print(homework2_data.head(10))\n",
    "\n",
    "# TODO: 在此完成作业2\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 作业3:电商销售EDA(综合)\n",
    "\n",
    "对某电商平台的销售数据进行完整EDA,数据包含商品、类目、地区、销售额、折扣等。要求:\n",
    "1. 使用本讲学习的EDA流程,完成完整分析\n",
    "2. 至少包含10个可视化图表\n",
    "3. 分析各类目、各地区的销售表现\n",
    "4. 分析促销活动的效果\n",
    "5. 识别异常订单\n",
    "6. 撰写完整的EDA报告(包含数据概览、关键发现、业务建议)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 作业3数据生成\n",
    "np.random.seed(42)\n",
    "n_orders = 2000\n",
    "\n",
    "homework3_data = pd.DataFrame({\n",
    "    '订单号': [f'O{str(i).zfill(6)}' for i in range(1, n_orders+1)],\n",
    "    '商品类目': np.random.choice(['电子产品', '服装', '食品', '家居', '图书'], n_orders, p=[0.25, 0.25, 0.2, 0.2, 0.1]),\n",
    "    '地区': np.random.choice(['华东', '华南', '华北', '西南', '东北'], n_orders, p=[0.3, 0.25, 0.2, 0.15, 0.1]),\n",
    "    '销售额': np.random.gamma(2, 300, n_orders).clip(50, 10000),\n",
    "    '数量': np.random.poisson(3, n_orders).clip(1, 20),\n",
    "    '是否促销': np.random.choice(['是', '否'], n_orders, p=[0.3, 0.7]),\n",
    "    '折扣率': np.random.uniform(0, 0.5, n_orders),\n",
    "    '客户评分': np.random.choice([1, 2, 3, 4, 5], n_orders, p=[0.02, 0.05, 0.15, 0.45, 0.33])\n",
    "})\n",
    "\n",
    "# 促销订单销售额提升\n",
    "homework3_data.loc[homework3_data['是否促销']=='是', '销售额'] *= 1.2\n",
    "homework3_data['销售额'] = homework3_data['销售额'].round(2)\n",
    "homework3_data['折扣率'] = homework3_data['折扣率'].round(3)\n",
    "\n",
    "print(\"作业3数据预览:\")\n",
    "print(homework3_data.head(10))\n",
    "\n",
    "# TODO: 在此完成作业3\n",
    "# 提示:可以使用本讲定义的generate_eda_report()函数作为起点\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## 课程结束\n",
    "\n",
    "**下一讲预告**: 第4阶段_第5讲_线性回归分析\n",
    "\n",
    "将学习:\n",
    "- 线性回归原理\n",
    "- 单变量和多变量回归\n",
    "- 模型评估指标\n",
    "- 回归诊断和假设检验\n",
    "- 实战案例:销售预测、价格预测\n",
    "\n",
    "---\n",
    "\n",
    "📧 如有疑问,请联系助教\n",
    "\n",
    "✅ 完成作业后,请提交Jupyter Notebook文件"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
